In the S/390 G5 storage controller and hierarchical memory subsystem, upto four independent storage banks have been implemented with each having its own dedicated memory controller and each memory controller having its own dedicated group of memory requestors and its own dedicated memory card which permits highly concurrent operation. In this type of system, the memory card was designed to serve as an S/390 main and expanded storage with both levels of memory being located on the same physical unit.
Between the memory requester and the memory controller of the system there exists an asynchronous protocol with regard to making requests to the memory controller which results in access commands being sent to the memory card attached to the memory controller. The memory controller for each storage bank thus provides an asynchronous grant response to its requester when the memory card resources become available to that requestor.
Between the memory controller and the memory card, there exists a well defined protocol logic dictating when the memory controller can send or receive data to and from the memory card. In many machines, like the G5, the memory card is also operating in a multiple memory bank configuration of its own(i.e can process multiple requests concurrently). Some past designs have had a "cancel mechanism" where the memory controller, upon receiving a cancel signal from the memory requestor, tells the memory card that the current request is being cancelled. However, even when DRAM access in the memory card can be suppressed by the early arrival of the cancel request, the memory card has already wasted some cycles of processing in addition to recovery time in order to be able to process the next access request. While this command was cancelled, any other requestor still has to wait until the bus protocol is no longer busy. This impacts memory utilization and throughput. If the late arrival of the cancel request prevented the suppression of the DRAM access, the memory card processing and recovery time is even longer impacting memory utilization and throughput to a greater extent.
In such systems, in order to maximize system throughput, we could have multiple requests into a memory controller one each from different ones of multiple memory requesters while simultaneously the memory requestors are accessing the Remote Cache. Then, there could be a situation where a request to the memory controller is no longer needed because the latest data were found in the Remote Cache or the Remote Cache rejected the request because of a conflict which requires the requestor to be recycled later on. With high cache hit ratios, there could be a significant impact to memory utilization and ultimately to throughput by the memory card processing the unwanted accesses.